close #66

Depends on this PR: lance-format/lance#5117
- New Public API: lance_ray.create_index, is introduced as the primary
entry point for building distributed vector indices, currently support
distributed IVF_FLAT, IVF_SQ, and IVF_PQ indices.

The new create_index function orchestrates a multi-phase workflow:
1. Global Training: It uses existing lance.IndicesBuilder to train IVF
centroids and, if applicable, PQ codebooks on a sample of the dataset.
2. Distributed Task Execution: Per-fragment index building tasks are
distributed across a pool of Ray workers. Each worker receives the
pre-trained models and processes a subset of the data fragments.
3. Metadata Finalization: After all fragment-level indices are built,
the main process merges the metadata and commits the new index to the
dataset manifest.

close lance-format#4723   

**Key Changes:**        
The distributed index creation leverages the existing IVF framework
while adding coordination mechanisms for multi-node execution. The index
merger component now handles distributed fragment consolidation and
metadata synchronization. This work enables scalable vector index
creation for large-scale datasets, significantly reducing index build
time.

- Implemented distributed IVF index building infrastructure for parallel
index construction across multiple nodes
- Enhanced the index merger component for distributed operations
- For IVF_HNSW part, the HNSW graph is built locally within the shard as
a sub-index of the partition; there is no cross-shard graph merging and
no cross-shard edges. These are supported but distribution only happens
in IVF.
- CPU only, torch accelerator will not be supported and fall back to
single node IVF index creation.

**Current Status in this PR:**
• FLAT/SQ: Should work now, it is under active testing phase, validating
distributed performance and accuracy
• PQ (Product Quantization): Currently depends on global training
codebook, requiring centralized training before distributed deployment.
• RQ (Residual Quantization): I didn't consider this when I design this
PR. Not yet supported in distributed mode maybe, planned for future
implementation

Once I finish all the testing phase in my side on performance and recall
accuracy, I will mark it ready to review.

---------

Co-authored-by: yanghua <yanghua1127@gmail.com>